Semi-Supervised Learning of Named Entity Substructure

The goal of this project was two-fold: (1) to provide an algorithm to correctly find and label named entities in text, and (2) to uncover substructure in the named entities (such as a first name, last name distinction among person entities). The underlying algorithm used is a Class Hidden Markov Model (CHMM), a Hidden Markov Model with hidden states that emit observed words as well as observed classes. This algorithm is further bolstered by incorporating features into the model, substituting the multinomial probability distributions for transitions and emissions in the model with the outputs of logistic regressions using the features.